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ABSTRACT 

Objectives: To identify and critically assess the extent 
to which systematic reviews of enhanced recovery 
programmes for patients undergoing colorectal surgery 
differ in their methodology and reported estimates of 
effect. 

Design: Review of published systematic reviews. We 
searched the Cochrane Database of Systematic Reviews, 
the Database of Abstracts of Reviews of Effects (DARE) 
and Health Technology Assessment (HTA) Database 
from 1990 to March 2013. Systematic reviews of 
enhanced recovery programmes for patients undergoing 
colorectal surgery were eligible for inclusion. 
Primary and secondary outcome measures: The 
primary outcome was length of hospital stay. We 
assessed changes in pooled estimates of treatment 
effect over time and how these might have been 
influenced by decisions taken by researchers as well as 
by the availability of new trials. The quality of 
systematic reviews was assessed using the Centre for 
Reviews and Dissemination (CRD) DARE critical 
appraisal process. 

Results: 10 systematic reviews were included. 
Systematic reviews of randomised controlled trials have 
consistently shown a reduction in length of hospital 
stay with enhanced recovery compared with traditional 
care. The estimated effect tended to increase from 
2006 to 2010 as more trials were published but has 
not altered significantly in the most recent review, 
despite the inclusion of several unique trials. The best 
estimate appears to be an average reduction of around 
2.5 days in primary postoperative length of stay. 
Differences between reviews reflected differences in 
interpretation of inclusion criteria, searching and 
analytical methods or software. 
Conclusions: Systematic reviews of enhanced 
recovery programmes show a high level of research 
waste, with multiple reviews covering identical or very 
similar groups of trials. Where multiple reviews exist 
on a topic, interpretation may require careful attention 
to apparently minor differences between reviews. 
Researchers can help readers by acknowledging 
existing reviews and through clear reporting of key 
decisions, especially on inclusion/exclusion and on 
statistical pooling. 



Strengths and limitations of this study 



» 



Systematic reviews of randomised trials have 
consistently shown that enhanced recovery pro- 
grammes reduce length of hospital stay for 
patients undergoing colorectal surgery, com- 
pared with usual care. The strength of this study 
is that we have looked in some detail at the avail- 
able reviews to identify differences between them 
and possible explanations, such as differences in 
intervention and outcome definitions and hand- 
ling of missing data from included trials in 
meta-analyses. 

We found a high level of research waste, with 
multiple reviews covering identical or very 
similar groups of trials. Differences in pooled 
effect estimates across reviews reflected differ- 
ences in interpretation of inclusion criteria, 
searching and analytical methods or software. 
Where multiple reviews exist on a topic, inter- 
pretation may require careful attention to appar- 
ently minor differences between reviews. 
Researchers can help readers by acknowledging 
existing reviews and through clear reporting of 
key decisions, especially on inclusion/exclusion 
and on statistical pooling. 
We identified limitations in reporting as one of 
the main barriers to understanding differences 
between reviews. These reporting issues often 
limited our ability to comment on whether deci- 
sions taken by review authors appear to be 
'right' or 'wrong' 



INTRODUCTION 

Reduction in length of stay in secondary care 
hospital settings provides a key potential 
opportunity to improve productivity in 
healthcare systems. There has been growing 
interest over recent years in the use of 
enhanced recovery programmes (also known 
as enhanced recovery after surgery (ERAS), 
fast track, multimodal, rapid or accelerated 
recovery programmes). The approach was 
pioneered in Denmark in the late 1990s for 
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patients undergoing colorectal surgery and is now 
spreading to other surgical pathways such as musculo- 
skeletal, urology and gynaecology. 

The underlying aim of enhanced recovery pro- 
grammes is to ensure that patients are in optimal condi- 
tion for treatment (to minimise the risk of surgery being 
postponed or cancelled because of the patient's condi- 
tion) , receive innovative care during surgery and experi- 
ence optimal postsurgical rehabilitation/ Programmes 
differ widely but share common elements such as 
patient education and involvement in preoperative plan- 
ning processes, preoperative oral carbohydrates, 
improved anaesthetic and postoperative analgesic techni- 
ques to reduce the physical stress of the operation, early 
oral feeding and mobilisation.^ ^ Enhanced recovery 
programmes have been delivered in the UK National 
Health Service (NHS) since the early 2000s. 
Implementation has to date been variable despite the 
support of the department of health and more recently 
the Royal Colleges. It is likely that this variation reflects 
the complexity of enhanced recovery programmes them- 
selves and issues around implementing change in funda- 
mental surgical procedures at a time when the NHS is 
facing severe funding constraints. 

There are a substantial number of systematic reviews 
and economic evaluations that examine the effectiveness 
and cost effectiveness of enhanced recovery pro- 
grammes. We have used this evidence as the basis of a 
comprehensive rapid evidence synthesis relating to the 
effectiveness, cost effectiveness, implementation, delivery 
and impact of enhanced recovery programmes with par- 
ticular reference to secondary care hospital settings in 
the NHS (Paton et al, in preparation) . During the course 
of this project, we became aware of significant methodo- 
logical differences between systematic reviews of 
enhanced recovery programmes in colorectal surgery 
(by far the largest body of evidence for any surgical 
specialty). Reviews published at around the same time 
varied in the trials they included and in their estimates 
(derived by meta-analysis) of the reduction in length of 
stay associated with enhanced recovery programmes. 

The objective of this study, therefore, is to provide a 
critical overview of the methodology of the available sys- 
tematic reviews and their contribution to the develop- 
ment of the evidence base available to decision-makers. 
In particular, we will examine how numerical estimates 
of the benefit of enhanced recovery on length of hos- 
pital stay have changed over time and how these might 
have been influenced by decisions taken by systematic 
reviewers as well as by the availability of new trials. 

METHODS 
Literature searches 

This study was carried out following a rapid synthesis of 
evidence relating to enhanced recovery programmes in 
all types of surgery (Paton et al, in preparation). To iden- 
tify systematic reviews, we searched the Cochrane 



Database of Systematic Reviews, the Database of Abstracts 
of Reviews of Effects (DARE) and Health Technology 
Assessment (HTA) Database from 1990 to March 2013. 
See online supplementary appendix 1 for search strat- 
egies. The PROSPERO database was searched to identify 
any ongoing systematic reviews. Systematic reviews evalu- 
ating enhanced recovery programmes in patients under- 
going any type of elective surgery in a hospital setting in 
the UK NHS or a comparable healthcare system were eli- 
gible for inclusion in the rapid evidence synthesis. Review 
authors' definitions of enhanced recovery programmes 
were accepted. Outcomes of interest were any measure of 
clinical outcomes, patient experience or resource use. 
Reviews had to compare enhanced recovery with usual/ 
standard care without a structured multimodal enhanced 
recovery pathway. For this methodological study, only sys- 
tematic reviews of enhanced recovery programmes for 
colorectal surgery were considered. 

Study selection 

We stored the literature search results in a reference 
management database (EndNote X6). Two researchers 
independently screened all titles and abstracts obtained 
through the searches for potentially relevant articles. 
Full manuscripts of potentially relevant articles were 
ordered and two researchers independently assessed the 
relevance of each article using the criteria stated above. 
Disagreements between researchers were resolved by dis- 
cussion or by recourse to a third researcher where 
necessary. 

Quality assessment 

Quality assessment of systematic reviews was based on 
the Centre for Reviews and Dissemination (CRD) critical 
appraisal processes for DARE (http://www.crd.york.ac. 
uk/crdweb/HomePage.asp). Specific aspects assessed 
were adequacy of the search; assessment of quality/ risk 
of bias of included studies; quality assessment results 
taken into account in the analysis; study details reported 
and differences between studies accounted for; investiga- 
tion of statistical heterogeneity; were gaps in research 
identified; and were the review conclusions justified. 
The quality assessment was performed by one researcher 
and checked by a second. Discrepancies were resolved 
by discussion or by recourse to a third researcher. 

Data extraction and analysis 

Clinical effectiveness data for the rapid evidence synthe- 
sis were extracted into review software (EPPI Reviewer 
V.4.0). An analysis plan for this methodological study 
was prepared in advance and additional methodological 
information was extracted into a Microsoft Excel spread- 
sheet. Data were extracted by one researcher and 
checked by another; discrepancies were resolved by con- 
sensus or where necessary by recourse to a third 
researcher. 

Data extracted from the reviews included the systematic 
review inclusion criteria ( particularly the definition of an 
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ERAS programme); randomised controlled trials (RCTs) 
included (number and list); length of stay mean difference 
(MD) estimates and 95% CIs for each RCT as reported in 
the review; meta-analysis methods (eg, type of model 
used) ; method used to handle missing means/ SDs; defin- 
ition of primary and total length of stay; pooled estimates 
of weighted MD (WMD) in length of stay between ERAS 
and control groups and source of funding. 

Extracted data were examined and tabulated to identify 
differences between reviews that may have influenced 
their conclusions and quantitative estimates of the effect- 
iveness of enhanced recovery programmes compared with 
usual care. We focused on the outcome of length of 
primary hospital stay (and total length of stay including 
readmissions, where reported) because reduction of 
length of stay is a key objective of enhanced recovery pro- 
grammes and because length of stay was a primary 
outcome of most included systematic reviews. Only length 
of stay data from RCTs was included in this analysis. 



RESULTS 

The report by Paton et al included 1 1 systematic reviews 
of ERAS programmes for colorectal surgery. One review^ 
focused on quality of life and patient satisfaction and 



another on compliance and variations in practice,^ 
leaving nine reviews that reported length of stay, of 
which seven reported a pooled effect estimate (WMD in 
days). The review by Zhuang et al,^ which was published 
too late to be fully discussed by Paton et al, is also 
included in this report, giving a total of 10 systematic 
reviews that met the inclusion criteria for this methodo- 
logical study (table 1). 

Most of the included reviews were reasonably well con- 
ducted and reported (table 2). The reviews by Wind 
et al^ Walter et al,^^ Spanjersberg et af^ and Zhuang 
et af met all seven quality criteria and were considered 
at low risk of bias. Three other reviews met six criteria 
but all failed to take study quality into account in their 
synthesis. Adamina et at^ and Lv et af^ failed to 
meet two of the criteria, while the paper by Rawlinson 
et at"^ only clearly met two criteria and was considered 
potentially at high risk of bias. 

Chronological development of the evidence base 

Four systematic reviews were published in the years 
2006-2009. At this stage, only a few randomised trials 
were available. Wind et al included three trials; 
Eskicioglu et al and Gouvas et al four trials and Walter 
et a/ included just two trials (table 1). 



Table 1 Overview of systematic reviews of ERAS for colorectal surgery 



Review 



Minimum definition of ERAS 
intervention 



Search cut-off 
date 



Included RCTs 



Wind et aF 
Eskicioglu etaP^ 
Gouvas etaP^ 
Walter et al^"^ 

I 

Varadhan etal^^ 

Adamina et aP^ 
Rawlinson et al^^ 

Spanjersberg et aP 

Lvef aF 

Zhuang et aP 



At least four elements required 
Not stated 

At least four elements required 

At least five elements required 
(preoperative, perioperative and 
postoperative) 

At least four elements required 
(preoperative, perioperative and 
postoperative) 

Documented compliance with at 
least four of five key elements 
At least four elements required 
(preoperative, perioperative and 
postoperative) 

At least seven elements required 



Not stated 



December 
2005 
May 2008 

July 2008 

January 2007 

November 
2009 

June 2010 
February 201 1 



Unclear 
(January 
2011?) 
April 2012 



At least seven elements required July 2012 



Anderson ef a/,^ Delaney ef a/,^ Gatt efa/^° 

Anderson ef a/,^ Delaney ef a/,^ Gatt ef a/,^° 
Khoo etal^^ 

Anderson etal,^ Delaney etal^ Gatt ef a/,^° 
Khoo etal^^ 

Anderson ef a/,^ Gatt ef a/^° 



Anderson ef a/,^ Delaney ef a/,^ Gatt ef a/,^° 
Khoo ef a/,^^ Muller etal,^^ Serclova ef a/,^^ 

Anderson ef a/,^ Delaney etal,^ Gatt ef a/,^° 
Khoo etal,^^ Muller ef a/,^^ Serclova eta^ 
Anderson ef a/,^ Delaney ef a/,^ Gatt ef a/,^° 
Khoo ef a/,^^ Muller ef a/,^^ Serclova eta^ 

Anderson ef a/,^ Gatt ef a/,^° Khoo ef a/,^^ 
Serclova ef aP 

Anderson ef a/,^ Delaney ef a/,^ Gatt ef a/,^° 
Khoo ef a/,^^ Muller efa/,^^ Serclova ef a/,^^ 
Vlug etaF 

Anderson ef a/,^ Gatt ef a/,^° Khoo ef a/,^^ 
Muller ef a/,"*^ Serclova ef a/,"*^ lonescu etal,^^ 
Vlug etai,^^ Garcia-Botello etal,^"^ 
van Bree ef a/,^^ Ren ef a/,^^ Wang G ef a/,^^ 
Wang Q efa/,^^ Yang etaF 



ERAS, enhanced recovery after surgery; RCT, randomised controlled trial. 
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Table ^H^^ ^^^^^^^^^^^^^^^^^^^^1^^^^^ 








Quality 


study details 














score 


reported and 








m 




Risk of 


accounted 


differences 


Statistical 


Gaps in 






Adequate 


bias 


for in 


accounted 


heterogeneity 


research 


Conclusions 1 


Review 


search 


assessed 


analysis 


for 


investigated 


identified 


justified 1 


Wind ef a/ 


/ 


/ 


/ 


/ 


/ 


/ 


/ J 


Eskicioglu etal^^ 


/ 


/ 


X 


/ 


/ 


/ 




Gouvas ef 


/ 


/ 


X 


/ 


/ 


/ 




Walter ef a/ 


/ 


/ 


/ 


/ 


/ 


/ 




Varadhan etaP^ 


✓ 


/ 

V 


V 

A 


✓ 


✓ 


/ 

V 




Adamina etaP^ 


/ 


/ 


uc 


/ 


UC 


/ 




Rawlinson etaP^ 


/ 


X 


X 


/ 


UC 


X 


UC 


Spanjersberg 


/ 


/ 


/ 


/ 


/ 


/ 




efa^° 
















Lvef aF^ 


/ 


/ 


X 


X 


/ 


/ 




Zhuang etaP 


/ 


/ 


/ 


/ 


/ 


/ 




DC, unclear. J 



Four further reviews were published in 2010 and 2011. 
These all considered the same six trials, although one 
review chose to exclude two of these from its main ana- 
lysis because the intervention did not meet the review 
definition of an ERAS programme. 

A systematic review by Lv a/ published in 2012 
added one more trial, bringing the total to seven. 
However, just a year later Zhuang a/ published a sys- 
tematic review with 13 included trials, among them 4 
recent trials by Chinese investigators. With one excep- 
tion,^^ the trials included in this review were identified 



and discussed by Paton a/ in our review of trials not 
included in the then-available systematic reviews. 

The included reviews varied somewhat in the extent to 
which they cited previously published reviews of the 
field. However, in general, reviews tended to cite most of 
the earlier reviews that would have been available at the 
time of writing. The first systematic review, by Wind et al, 
was cited by eight of the nine subsequent reviews. 
Gouvas et aFs review was cited by six of seven subsequent 
reviews and Varadhan et al by four out of five (table 3) . 
The main exception was the paper by Rawlinson et al}^ 



Table 3 Cross-citation among included systematic reviews 



Review 



Reviews cited 



Cited by subsequent reviews 



Wind et aF 



Eskicioglu eta^ 
Gouvas etaP 



Walter efa/^^ 

Varadhan etal^^ 

Adamina et al^^ 
Rawlinson et al^^ 

Spanjersberg et aP 

Ly etaP 

Zhuang etaP 



NA 



Wind et aF 
Wind et aF 



Wind et aF 

Wind etalj Eskicioglu ef a/,^^ Gouvas etal,^^ 
Walter etal^"^ 

Wind etalj Gouvas ef aP 

Wind etalj Eskicioglu etal,^^ Gouvas ef a/,^^ 

Varadhan etaf^ 

Wind etalj Gouvas ef a/,^^ Walter ef a/,^"^ 
Varadhan etal^^ 

Gouvas ef a/,^^ Walter ef a/,^"^ Varadhan ef a/,^^ 
Spanjersberg ef a^° 

Wind etalj Eskicioglu ef a/,^^ Gouvas ef a/,^^ 
Walter ef a/,^"^ Varadhan ef a/,^^ 
Adamina etal,^^ Spanjersberg etaF^ 



Eskicioglu etal,^^ Gouvas ef a/,^^ Walter ef a/,^"^ 
Varadhan etal,^^ Adamina etal,^^ 
Rawlinson etal,^^ Spanjersberg ef a/,^° 
Zhuang etaP 

Varadhan etal,^^ Rawlinson etal,^^ Zhuang etaP 
Varadhan etal,^^ Adamina etal,^^ Rawlinson 
etal,^^ Spanjersberg ef a/,^° Lv et al,^^ 
Zhuang etaP 

Varadhan etal,^^ Spanjersberg ef a/,^° Lv etal,^^ 
Zhuang etaP 

Rawlinson etal,^^ Spanjersberg etal,^^ Lv etal, 
Zhuang et aP 
Zhuang etaP 
Not cited 

Lv et al,^^ Zhuang et aP 
Not cited 
NA 
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NA, not applicable. 
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Systematic review 

Wind 2006 
Gouvas 2009 
Walter 2009 
Varadhan 2010 
Spanjersberg 2011 
Adamina 2011 
Lv2012 
Zhuang 2013 



WMD (95% CI) 

1.89 (0.18 . 3.61 ) 

1.88 (0.41 . 3.35 ) 

3.64 (2.29.4.98 ) 

2.51 ( 1.47.3.54 ) 

2.94 (2.19.3.69 ) 

2.50 ( 1.11 . 3.92 ) 

1.88 (0.86.2.91 ) 

2.44 ( 1.83,3.06) 



1 1 f- 

0 12 3 

Reduction in length of stay (days) 



Figure 1 Summary of pooled results for primary length of 
stay (WMD, weighted mean difference). 



which was not cited by three later reviews. Adamina et al 
differed from other researchers in their use of Bayesian 
methodology and their work was only cited by one of 
the four later reviews. Zhuang et al cited all the previous 
systematic reviews except that of Lv et al, which may have 
been published too late to be included. 



Methodological differences between systematic reviews 

The four early systematic reviews of ERAS for colorectal 
surgery showed a number of methodological differences. 
Although inclusion criteria appeared similar, Walter et al 
excluded two trials that were included in the reviews by 
Eskicioglu et al and Gouvas et al. In one case,^ this was 
reported to be because the trial included some patients 
who had undergone small bowel surgery; the reason for 
the other exclusion^^ was not reported. Walter et al also 
differed from the other reviews in its definition of out- 
comes: length of stay was measured as total days of 
admission in this review and as days spent in hospital 
after surgery in the other reviews published in the same 
period. 

In terms of data synthesis, Eskicioglu et al was the only 
one of the four early reviews that did not perform a 
meta-analysis, on the grounds that data in the right form 
(means and SDs) were not available. The remaining 
three reviews elected to pool. Gouvas et al and Wind et al 
used random effects models for their main meta- 
analysis, while Walter et al used a fixed effects model. 
Wind et al and Walter et a/ justified their choice on the 
basis of a random effects model being more appropriate 
in the presence of some heterogeneity but neither 
stated a level of heterogeneity above which a random 
effects model should be used. Wind et al stated that trials 
without means and SDs were omitted from the analysis, 
while Gouvas et al estimated from reported medians and 



ranges where necessary. Walter et al did not report on 
this point. 

Pooled estimates (WMD) of the effect of ERAS on 
primary length of stay varied substantially, being consid- 
erably higher and with a wider CI in Walter et al com- 
pared with the other reviews (figure 1 and table 4). 
Wind et al and Gouvas et al provided almost identical 
WMDs, the only effect of an additional trial in Gouvas 
et al being to slightly narrow the 95% CI. Walter et al 
reported no statistical heterogeneity while the other two 
reviews found some evidence of heterogeneity. 

Overall, these reviews suggested considerable uncer- 
tainty about the magnitude of the effect of ERAS pro- 
grammes in reducing length of hospital stay. Based on 
95% CIs, the effect could plausibly range from less than 
0.5 to almost 5 days. 

The next group of systematic reviews to be published 
15 18 19 20 included up to six RCTs (additional RCTs by 
MuUer et al^^ and Serclova et al^^ were now available). 
Spanjersberg et al differed from other authors in treating 
length of stay as a secondary outcome of the review and 
more significantly by excluding two trials that were 
included in most other systematic reviews. The authors 
stated that Muller et al and Delaney et af failed to meet 
the inclusion criterion of including at least seven ele- 
ments in the ERAS intervention. 

None of these reviews provided clear definitions of out- 
comes but Varadhan et a/ and Spanjersberg et a/ appeared 
to measure days in hospital after surgery (ie, primary 
length of stay) . Adamina et al reported their outcome as 
length of hospital stay without definition or distinguish- 
ing between primary and total length of stay. The reviews 
also differed in their reported treatment of studies with 
missing data (mean/SD): Varadhan et al obtained data 
from original authors, Spanjersberg et al calculated from 
median and range and Adamina et al did not report their 
methods. Adamina et al were the only authors who used 
Bayesian methods for meta-analysis, although few details 
were reported. Rawlinson et al^^ did not report an ori- 
ginal meta-analysis but instead discussed the findings of 
Gouvas et al and Varadhan et al. 

Compared with the earlier reviews, this group of sys- 
tematic reviews reported a substantially narrower range of 
effect estimates, all suggesting a reduction of 2.5-3 days 
in primary hospital stay associated with ERAS pro- 
grammes (table 1). The range of 95% CIs was also nar- 
rower (approximately 1.1-3.9 days). 

The 2012 review by Lv et af^ added one additional 
RCT for a total of seven trials. The new trial^^ compared 
ERAS with traditional care in patients undergoing lap- 
aroscopic and open surgery and so was treated as two 
trials in the meta-analysis. The trial showed a 1-day 
reduction in length of stay in the laparoscopic setting, 
but there was no difference in patients undergoing open 
surgery. Lv et al also differed from most of the earlier 
reviews in defining length of stay as length of the index 
admission (rather than days in hospital after surgery). 
It is unclear whether this explains the discrepancy but 
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Lv et al reported a smaller reduction in length of stay 
compared with the reviews immediately preceding it 
(-1.88 days, 95% CI -2.91 to -0.86). However, this esti- 
mate was still compatible with a 'true' reduction of 
around 2-3 days in primary length of stay. 

The most recent systematic review that we are aware of 
was published in May 2013, too late to be considered in 
detail in the full report.^ It is more comprehensive than 
the previous reviews, with 13 included trials (two with 
open and laparoscopic arms) and 1910 participants. All 
except one^^ of the RCTs included in our report were 
also in this systematic review. However, Zhuang et al also 
included one trial that we excluded. In fact, this publi- 
cation is a single-centre report of results from the laparos- 
copy and/or fast track multimodal management versus 
standard care (LAFA) trial that was reported in full by 
Vlug et al^^ so it is likely that there was double counting 
of these patients in Zhuang et aFs meta-analyses. 
However, this publication was not included in the 
meta-analysis of length of stay outcomes. 

Some of the trials included for the first time in 
Zhuang et aFs review, particularly that of lonescu et al^^ 
were available to earlier systematic reviewers. Of the 
reviews published in 2010 or later, Varadhan et a/ did not 
give references for any excluded studies. Adamina et al 
gave references for excluded studies but did not list 
lonescu et al, suggesting that their search did not locate 
this trial. Rawlinson et al did not list excluded studies, 
while the Cochrane review by Spanjersberg et al did not 
list lonescu et al among the excluded studies. Lv et al 
excluded 26 out of 33 studies examined in full text but 
did not report any details. 

There was a discrepancy between Zhuang et al and 
Lv et al in their data extraction of the trial by Vlug et al. 
There were also minor data extraction discrepancies 
between these reviews for two other trials, but as they 
were in opposite directions in the two cases, this is 
unlikely to have had a major impact on the overall 
meta-analysis results. 

Like most included reviews, Zhuang et al defined 
length of stay as time in hospital after surgery. The 
pooled estimate of a 2.44 day reduction in primary 
length of stay (95% CI 1.83 to 3.06) for ERAS relative to 
traditional care was very similar to those reported by 
Varadhan et al^^ and Adamina et al}^ 



DISCUSSION 
Main findings 

Systematic reviews of randomised trials have consistently 
shown a reduction in length of hospital stay with ERAS 
compared with traditional care. The estimated effect 
tended to increase over time from 2006 to 2010 as more 
trials were published but has not altered significantly in 
the most recent review, despite the inclusion of several 
unique trials. The best estimate appears to be an average 
reduction of around 2.5 days in primary postoperative 
length of hospital stay. However, there is considerable 
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heterogeneity in most pooled estimates and the com- 
plexity of ERAS as an intervention means that the bene- 
fits achieved in routine practice may differ from those 
reported in clinical trials. These issues are discussed in 
the full project report (Paton et al, in preparation). 

The published systematic reviews show a high level of 
redundancy, with multiple reviews covering identical or 
very similar groups of trials. Differences in pooled effect 
estimates across reviews reflect differences in interpret- 
ation of inclusion criteria, searching and analytical 
methods or software. A few data extraction discrepancies 
were observed but these are unlikely to have significantly 
affected review conclusions. 

Findings in relation to previous studies 

The existence of multiple systematic reviews covering the 
same topic was investigated by Siontis et al^^ They found 
that of 73 systematic reviews (with meta-analysis) pub- 
lished in 2010, 49 (67%) had at least one other pub- 
lished meta-analysis on the same topic. A particularly 
striking example was the existence of 11 meta-analyses of 
statins to prevent atrial fibrillation after cardiac surgery; 
all except the first of these showed a large positive effect 
of the intervention. Our findings reinforce those of 
Siontis et al, although even where reviews included 
exactly the same trials, their pooled estimates of effect 
were not necessarily identical. Enhanced recovery differs 
from an intervention like statins after cardiac surgery, 
being more complex and changing over time as more 
elements are incorporated into routine practice. This 
would tend to favour increased variation between 
reviews along the lines we observed. 

A slightly different approach to investigating overlap 
between systematic reviews was taken by Woodman 
et al^^ who looked at the differences between eight 
reviews of community interventions to promote physical 
activity. Across the eight reviews, there were 28 included 
studies, of which 22 (79%) were only included in one 
review. There was little cross-citation between reviews. 
For most reviews. Woodman et al could explain why 
primary data were not included, which was usually due 
to the reviews having a relatively narrow scope. Despite 
these issues, the review conclusions were similar. 
Comparing our results with those of Woodman et al, the 
enhanced recovery reviews had a higher degree of 
overlap, although they differed with respect to inclusion 
of non-randomised studies (not discussed in this paper) . 
The differences between the enhanced recovery and 
physical activity reviews probably reflect the fact that 
enhanced recovery, although a complex intervention, is 
more narrowly defined than interventions to increase 
physical activity. 

Implications 

The findings of this study have implications for system- 
atic reviewers, readers of systematic reviews and produ- 
cers of systematic review-based evidence products. 
Systematic reviewers can help readers compare their 



reviews with other similar studies by clear reporting of 
key decisions, especially on inclusion/exclusion and on 
statistical pooling. Most of the reviews of enhanced 
recovery programmes performed a meta-analysis to cal- 
culate a WMD in length of stay between enhanced recov- 
ery and usual care groups. Calculating the WMD 
requires the means and SDs of length of stay in each 
group in the included trials, but trials often report a dif- 
ferent measure such as median and range (or IQR). 
Systematic reviews can deal with this situation in differ- 
ent ways, for example, by using formulae to estimate the 
mean and SD, contacting the trial authors or omitting 
trials from any meta-analysis if mean and SD are not 
reported. The enhanced recovery reviews adopted all of 
these approaches but did not report which data were 
estimated rather than derived directly from trial reports. 
Improved reporting in this area would improve transpar- 
ency and help readers to understand discrepancies 
between apparently similar meta-analyses. Length of stay 
outcomes was not always clearly defined or reported and 
differences between reviews may in part stem from the 
fact that they were measuring an outcome in different 
ways. 

Readers of systematic reviews need to be aware of the 
existence of multiple systematic reviews for a high per- 
centage of topics. This means they should not rely 
uncritically on the first review they find and underlines 
the importance of services that critically appraise system- 
atic reviews and those that provide overviews of reviews 
across a topic area. As an aid to transparency, 

authors of new systematic reviews should also acknow- 
ledge the existence of any systematic reviews addressing 
the same or a similar question. 

The use of systematic reviews to produce rapid evi- 
dence summaries to inform decision-making is an area 
where methodology is still developing, although some 
methodological frameworks have been published.^^ 
Assessment of the quality and reliability of systematic 
reviews is an important part of this process, but the 
present study suggests that conventional approaches to 
critical appraisal are not sufficient and careful attention 
should be paid to apparently minor differences between 
reviews. This may require increased extraction of meth- 
odological data, although the results may not be 
included in published evidence summaries aimed at 
decision-makers. 

Strengths and limitations 

The strength of this study is that we have looked in some 
detail at reviews of a complex clinical intervention to 
identify differences between them and possible explana- 
tions, such as differences in intervention and outcome 
definitions and handling of missing data from included 
trials in meta-analyses. We have identified limitations in 
reporting as one of the main barriers to understanding 
differences between reviews of the same topic. These 
reporting issues have often limited our ability to 
comment on whether decisions taken by review authors 
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appear to be 'right' or 'wrong'. The discrepancies 
between reviews identified in this study sometimes arose 
from decisions where both options could be considered 
reasonable, for example, whether or not to exclude the 
trial by Delaney et aP because a small number of patients 
undergoing small bowel surgery were included. 

This study only considered length of stay outcomes. 
Length of stay was selected as the primary outcome 
because of its importance for the health service at a 
time of acute financial pressure and because it was an 
outcome considered in all the included systematic 
reviews. Other important outcomes, including morbidity 
and mortality, are discussed in the full report (Paton 
et al, in preparation) . 
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Unanswered questions/further research 

The current study suggests a need for further, more 
in-depth research into methods of quality assessment of 
systematic reviews; to make the best use of a cumulative 
evidence base for decision-making and to identify meth- 
odological issues and decision points that may influence 
the eventual conclusions of a review. These issues are 
particularly important for researchers seeking to help 
decision-makers interpret and use systematic reviews. 
Enhanced recovery is a complex intervention with mul- 
tiple components and its successful implementation is 
likely to be influenced by numerous background factors. 
Given this background, researchers and clinicians carry- 
ing out new systematic reviews should ensure that their 
chosen method of synthesis is appropriate for exploring 
intervention complexity. The recently published 
research agenda for reviews of complex interventions^^ 
provides timely guidance in this regard. Ideally, system- 
atic reviews of emerging complex interventions (eg, 
interventions to support integration of health and social 
care) should use standardised methods and outcome 
definitions and be regularly updated, although this may 
be difficult to achieve in practice. 

Systematic reviews of enhanced recovery programmes 
in colorectal surgery continue to be published and we 
are aware of at least two publications since our search 
was completed. The authors of one of these reviews 
refer to their paper as 'a substantial update from previ- 
ous meta-analyses'^^ but in fact the included trials and 
overall findings are almost identical to those of Zhuang 
et al.^ This is not the fault of the authors but represents 
an avoidable waste of research resources. Systematic 
reviews are now increasingly being registered prospect- 
ively at the outset on databases such as PROSPERO^^ 
and it will be interesting to see whether the production 
of overlapping reviews decreases over time. 
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